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Executive  Summary 
TEST  EQUIPMENT  MANAGEMENT 


LMI 


The  Department  of  Defense's  (DoD's)  investment  in  fielded  test  equipment 
(manual  and  automatic  test  equipment  and  test  program  sets)  exceeds  $30  bil¬ 
lion  and  is  increasing  rapidly. 

The  problems  associated  with  acquiring  and  supporting  test  equipment  are 
varied  and  significant.  Much  of  it  does  not  work  as  well  as  expected,  is  both 
difficult  and  costly  to  support,  and  is  not  suitable  for  its  planned 
operational  environment.  For  some  test  equipment,  the  Military  Departments 
lack  the  assets  necessary  to  support  peacetime  operating  tempos,  let  alone 
those  expected  during  wartime.  The  consequences  go  beyond  the  dollars  already 
invested  in  test  equipment  and  the  annual  cost  of  its  support:  weapon  system 
readiness  suffers  from  inaccurate,  delayed,  or  prolonged  testing;  and  depot 
maintenance  workloads  are  increased  unnecessarily  with  field-reparable  modules 
and  components  because  test  equipment  in  the  field  is  neither  available  nor 
operational. 

Since  the  early  1970's,  the  Office  of  the  Secretary  of  Defense  (OSD)  and 
the  Military  Departments  have  sponsored  numerous  studies  of  test  equipment. 
Those  studies  have  repeatedly  documented  the  problems  and  identified  the 
solutions.  Some  solutions  implementable  through  the  acquisition  process  are 
being  pursued:  greater  emphasis  is  being  placed  on  support  during  develop¬ 
ment;  procurement  of  test  equipment  is  being  delayed  until  weapon  system 
designs  stabilize;  and  standardized  automatic  test  equipment  is  being  devel¬ 
oped.  However,  those  solutions  that  require  changes  in  the  management  and 
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support  of  test  equipment  after  it  enters  the  inventory  are  not  receiving 
similar  attention. 

It  is  time  for  the  OoD  to  take  the  actions  that  are  fundamental  to  an 
effective  test  equipment  management  program.  OSD  must  curb  the  proliferation 
of  many  different  types  of  manual  test  equipment;  improve  the  reliability  and 
maintenance  support  of  automatic  test  equipment;  and  shorten  run  times  and 
improve  the  diagnostic  performance  of  test  program  sets. 

We  believe  that  the  Assistant  Secretary  of  Defense  (Manpower,  Installa¬ 
tions,  and  Logistics)  must  be  the  catalyst  to  effect  these  and  other  actions 
within  the  DoD.  We  recommend  he  charge  the  Director,  Maintenance  Policy,  to 
take  the  following  actions  to  correct  longstanding  problems  in  test  equipment 
management : 

-  Develop  a  DoD-wide  preferred  items  list  for  manual  test  equipment; 

-  Institute  reporting  of  automatic  test  equipment  performance  and  avail¬ 
ability; 

-  Establish  procedures  for  collecting  test  program  set  run  times  and 
performance  data; 

-  Draft  a  DoD  instruction  that  prescribes  policy  and  procedures  for 
managing  and  supporting  test  equipment. 

These  actions  constitute  a  small,  first  step  toward  improving  management 
and  support  of  test  equipment  within  the  DoD.  However,  they  will  signal  the 
Military  Departments  that  the  problems  with  fielded  test  equipment  must  be 
corrected  and  that  OSD  intends  to  lead  the  way. 
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1.  BACKGROUND 


Since  the  early  1970’s,  the  Office  of  the  Secretary  of  Defense  (OSD),  the 
Military  Departments,  and  industry  have  sponsored  numerous  studies  of  test 
equipment,  formed  several  joint  panels  to  investigate  selected  technical 
issues  in  more  detail,  and  initiated  a  variety  of  programs  to  correct  identi¬ 
fied  problems.  Despite  such  attention  over  many  years,  the  Department  of 
Defense  (DoD)  still  faces  many  significant  problems  with  its  test  equipment. 
Most  of  the  problems  are  technical  in  nature;  several,  however,  can  be  traced 
directly  to  shortfalls  in  management.  Throughout  this  report,  we  make  exten¬ 
sive  use  of  previous  studies  of  test  equipment  to  draw  conclusions  about  those 
problems  and  shortfalls  and  to  identify  areas  in  which  increased  OSD  partici¬ 
pation  would  enhance  the  development  of  permanent  solutions. 

This  chapter  defines  the  terminology  used  throughout  the  report,  provides 
an  estimate  of  the  value  of  DoD's  inventory  of  test  equipment,  and  describes 
the  organization  of  the  balance  of  the  report. 

TERMINOLOGY 

Each  of  the  Military  Departments,  as  well  as  OSD,  has  adopted  its  own 
nomenclature  for  test  equipment.  Even  though  the  Army  has  an  official  defini¬ 
tion  for  "test  equipment,"  it  seldom  uses  that  term.  The  term  most  frequently 
used  by  the  Army  is  "test,  measurement  and  diagnostic  equipment  (TMDE),”  and 
it  is  defined  below. 

Any  system  or  device  used  to  evaluate  the  operational 
condition  of  an  end  item  or  subsystem  thereof,  or  to 
identify  and/or  isolate  any  actual  or  potential  malfunc¬ 
tion.  TMDE  includes  diagnostic  and  prognostic  equipment, 


semiautomatic  and  automatic  test  equipment  (with  issued 
software),  and  calibration  test  or  measurement  equipment.1 

The  Navy  considers  "electronic  test  equipment  (ETE)"  to  include  both 
manual  and  automatic  test  equipment  (ATE) .  It  further  categorizes  manual  ETE 
either  as  general  or  special  (i.e.,  designed  to  support  a  particular  weapon 
system)  purpose. 

The  Air  Force  treats  test  equipment  as  just  one  category  of  support 

equipment,  as  illustrated  by  the  following  definition. 

Support  equipment  includes  all  supporting  equipment  except 
equipment  that  is  an  integral  part  of  the  mission  equip¬ 
ment.  It  includes:  tools;  test  equipment;  ATE;  organiza¬ 
tional,  field,  and  depot  support  equipment;  related 
computer  programs  and  software.2 

For  the  most  part,  however,  the  Air  Force  refers  to  manual  test  equipment, 
ATE,  and  associated  software  jointly  as  precision  measurement  equipment,  a 
term  unique  to  the  Air  Force. 

The  DoD  does  not  define  either  test  equipment  or  TMDE,  but  does  use  the 

term  "support  and  test  equipment,"  as  defined  below,  in  the  budget  process. 

.  .  .  development  and  procurement  of  peculiar  support  and 
test  equipment  (e.g.,  test  program  sets)  and  major  items 
of  common  ST&E  [support  and  test  equipment]  (automated 
test  station,  fuel  storage,  and  handling  equipment,  etc.) 
required  for  the  new  system.3 

Each  of  the  Military  Departments  considers  ATE  to  be  essentially  the  same 
thing: 

Equipment  that  is  designed  to  conduct  analysis  of  func¬ 
tional  or  static  parameters  to  evaluate  the  degree  of 


^Department  of  the  Army,  "Test,  Measurement  and  Diagnostic  Equipment," 
Army  Regulation  750-43,  1  April  1984. 

2 

Department  of  the  Air  Force,  "Acquisition  of  Support  Equipment," 
Air  Force  Regulation  800-12,  20  May  1974. 

3 

Office  of  the  Secretary  of  Defense,  Program  Objectives  Memorandum  (POM) 
Preparation  Instructions,  Format  V-E-l  Attachment,  "Support  Element  Defini¬ 
tions,"  1984. 
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performance  degradation  and  may  be  designed  to  perform 
fault  isolation  of  unit  malfunctions.  The  decision 
making,  control,  or  evaluative  functions  are  conducted 
with  minimum  reliance  on  human  intervention. 4 

An  ATE  station  comprises  numerous  stimuli  and  measurement  instruments  inter¬ 
faced  with  and  under  the  control  of  a  computer  and  supplemented  with  various 
input/output  peripherals  (e.g.,  display,  printer,  control  station).  Some  of 
the  instruments  integral  to  the  ATE  may  be  identical  to  ETE  used  manually. 
The  ATE  examines  a  particular  item  (known  as  the  "unit  under  test”  or  UUT)  by 
executing  a  test  program  set  (TPS)  designed  for  that  item. 

Each  Military  Department  uses  the  term  TPS,  but  the  terminology  for  the 
elements  constituting  a  TPS  differs.  The  term  TPS  does  not  appear  in  MIL- 
STD-1309B  but  is  defined  in  a  Navy  Military  Standard"*  as  consisting  of  those 
items  necessary  to  test  a  particular  unit  using  ATE.  Those  items  include  the 
test  program  (a  coded  sequence  for  automatically  determining  the  operational 
readiness  condition  of  the  UUT) ,  interface  device  (the  connector  between  the 
ATE  and  UUT),  test  program  instruction  (the  additional  operator  instructions 
needed  to  carry  out  the  test) ,  and  supplementary  data  (the  additional  tech¬ 
nical  data  needed  for  the  test) . 

With  the  above  definitions  as  a  background,  the  term  "test  equipment" 
throughout  this  report  refers  to  the  composite  of  ETE  (limited  to  manual  test 
equipment  only,  including  both  general  and  special  purpose),  ATE,  and  TPSs. 
IMPORTANCE 


The  need  for  effective  test  equipment  management  within  the  DoD  is  best 
illustrated  by  the  DoD's  already  substantial  investment  in  test  equipment. 


4 

"Definition  of  Terms  for  Test,  Measurement  and  Diagnostic  Equipment,” 
Military  Standard  (MII-STD)  1309B,  30  May  1975. 

"*"Test  Program  Sets;  General  Requirements  for,"  MIL-STD-2077(AS) ,  9  March 

1978. 
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able  1-1  shows  that  the  DoD  has  invested  approximately  $30  billion  in  test 
quipment,  including  more  than  $14  billion  in  ATE  alone.  It  has  in  excess  of 
0,000  different  makes  and  models  of  ETE  (mostly  in  the  Air  Force)  and 
4,000  different  TPSs  (again,  mostly  in  the  Air  Force). 

TABLE  1-1.  DoD’s  INVENTORY  AND  INVESTMENT  IN  TEST  EQUIPMENT 


TEST 

EQUIPMENT 

ARMY 

NAVY 

AIR  FORCE 

TOTAL 

ETE 

Line  Items1 
Density2 

Cost3 

18,000 

730,000 

$3B4 

29,000 

700,000 

$2.8B 

54,000 
800,000 
$3. 7B 

80,000+ 

2.2M4 

$9.5B 

ATE 

Line  Items 
Density 

Cost 

110 

N/As 

$1B 

352 

N/A 

$3.5B 

425 

3,400 

$10B 

880+ 
3,400+ 
$14. 5B 

TPS 

Line  Items 

Cost 

1,000 

0.5B 

5,000 

$2.5B 

8,000 

$4B 

14,000+ 

$7B 

Total  Cost 

$31B 

^Number  of  unique  items  or  test  programs  as  identified  by  stock 


number  or  computer  program  index  number. 

2 

Total  number  of  items  in  the  inventory. 

3 

Initial  investment  including  development  and/or  acquisition. 

4 

B  =  billion;  M  =  million. 

~*Not  available. 


The  ETE  data  shown  in  Table  1-1  include  only  general-purpose  equipment 
(which  is  categorized  in  Federal  Supply  Class  (FSC)  6625).  The  number  of  ETE 
line  items  was  taken  from  a  recent  DoD-wide  report  on  the  status  of  stand¬ 
ardization  for  all  ETE  in  FSC  6625.^  All  of  the  ETE  cost  data  are  approxima¬ 
tions  because  that  information  is  not  available. 

^United  States  Army  Communications-Electronics  Command,  "Standardization 
Program  Analysis,  FSC  6625,"  Department  of  the  Army,  29  June  1984. 
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Each  Military  Department  knows  the  number  of  different  types  of  ATE  in 
s  inventory,  but  only  the  Air  Force  knows  how  many  it  has  of  each  type.  The 
my  monitors  closely  its  general-purpose  ATE  (such  as  the  AN/  USM-410,  Elee- 
onics  Quality  Assurance  Test  Equipment  —  EQUATE),  but  has  difficulty 
eping  track  of  special-purpose  ATE.  The  Navy's  situation  is  similar  to  that 
the  Army.  Furthermore,  the  ATE  cost  data  are  rough  estimates  only,  taken 
irgely  from  the  literature. 

The  number  of  different  types  of  TPSs  shown  in  Table  1-1  is  the  order  of 
ignitude  as  of  the  end  of  Fiscal  Year  (FY)  1984.  The  TPS  costs  are  estimates 
•cause  the  Military  Departments  do  not  keep  track  of  that  data.  The  esti- 
jtes  shown  are  based  on  a  factor  of  $500,000  for  each  unique  TPS.  Actual 
osts  of  currently  fielded  TPSs,  based  on  contract  data,  range  from  a  low  of 
30,000  to  a  high  of  $2.5  million,  depending  on  UUT  size  and  complexity,  and 
he  compatibility  of  the  UUT  with  the  ATE.  The  contract  costs,  however,  do 
ot  include  the  cost  of  developing  TPS  requirements  specifications,  the  cost 
f  spare  UUTs  for  TPS  validation  and  verification,  the  cost  of  ATE  time  needed 
or  TPS  development  and  verification,  the  cost  of  engineering  changes  to 
ccommodate  UUT  or  ATE  changes  or  to  improve  TPS  performance,  or  the  cost  of 
oftware  maintenance  and  configuration  management.  Consequently,  the  TPS  cost 
stimates  are  probably  conservative.  (TPS  density  is  not  shown  in  Table  1-1 
ecause  copies  can  be  made  of  unique  TPSs  at  relatively  small  cost.) 

Not  only  does  the  DoD  have  a  substantial  investment  in  test  equipment, 
ut  that  investment  is  estimated  to  be  growing  by  several  billion  dollars 
nnually. 

EPORT  ORGANIZATION 

Chapter  2  provides  an  overview  of  the  problems  with  test  equipment.  A 
ynopsis  of  numerous  initiatives  to  correct  the  problems  with  test  equipment 


presented  in  Chapter  3.  Chapter  4  identifies  several  areas  requiring 
ceased  participation  from  OSD  and  recommends  that  OSD  undertake  several 
cific  actions. 


Vertical  Testability.  The  term  "vertical  testability"  refers  to  the 

ent  to  which  tests  and  test  results  are  consistent  among  the  three  levels 

maintenance:  organizational  level  (system  BIT),  intermediate  level 

imarily  line-replaceable  unit  testing),  and  depot  level  (shop-replaceable 

t  testing).  The  manifestations  of  testability  problems,  such  as  high 

innot  duplicate"  and  "retest  okay"  (RTOK)  rates,  traditionally  have  been 

:ributed  to  system  BIT  shortfalls.  However,  recent  Air  Force  studies  of 

ill  BIT  indicate  that  about  50  percent  of  the  line-replaceable  units  testing 

>od"  on  ATE  are  actually  faulty,  confirming  the  accuracy  of  aircraft  BIT 
4 

lications.  The  F- Ill's  ATE  evidently  cannot  simulate  the  full  spectrum  of 
t  UUT's  operational  environment.  If  the  ATE  is  supplemented  with  "dynamic 
sters”  (i.e.,  hot  mockups),  then  most  vertical  testability  problems  dis¬ 
pear.  This  example  of  inadequate  vertical  testability  applies  to  other 
apon  systems  as  well  and  may  explain  many  of  the  unsatisfactory  experiences 
th  ATE. 

ST  PROGRAM  SETS 

The  problems  with  TPSs  are  straightforward:  they  seldom  are  developed  by 
e  time  the  weapon  system  is  fielded  and  when  they  become  available,  they 
ther  are  not  as  effective  as  planned  or  require  major  improvements  within 
e  first  2  years.  (The  Air  Force  estimates  that  40  percent  of  its  TPSs  have 

jor  deficiencies  when  they  are  fielded  and  that  90  percent  must  be  modified 

thin  2  years  of  fielding. )^  Furthermore,  the  Military  Departments  fre- 

ently  lack  the  technical  data  for  supported  systems  to  make  needed  TPS 
provements.  As  a  result,  excessive  delays  in  updating  and/or  improving  TPSs 

4 

"Enhancing  the  F— 111  Avionics  Intermediate  Shop  with  Dynamic  Test  Sta- 
ons,"  presented  by  Alton  E.  Patterson,  Anthony  Carnciro,  and  Eugene  M.  Long 
AUTOTESTCON  '83. 

^Air  Force  Logistics  Command,  "A  Study  of  Embedded  Computer  Support 

stems,"  September  1980. 
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incept  for  most  fielded  ATE  is  based  on  BIT  fault  isolating  to  a  tester- 
iplaceable  unit,  which  is  either  repaired  on-site  or  evacuated  to  a  higher- 
»vel  maintenance  facility.  Because  of  the  high  parts  count  of  large-scale 
rE,  the  random  behavior  of  failures  in  electronics  equipment,  the  high  reli- 
sility  of  individual  components,  and  the  low  density  of  identical  ATE  at  any 
ait,  the  demands  for  ATE  spares  and  repair  parts  seldom  meet  demand  stockage 
riteria,  so  that  supply  support  f'-r  ATE  is  frequently  inadequate.  The  Army's 
olution  is  to  authorize  an  essential  repair  parts  stockage  list  for  ATE, 
hile  the  Navy  and  Air  Force  always  deploy  their  ATE  in  pairs  (or  more), 
ermitting  the  cannibalization  of  one  ATE  to  keep  the  other(s)  operational, 
he  latter  approach  is  acceptable  in  peacetime  (cost  studies  have  actually 
hown  that  a  complete  standby  ATE  is  often  cheaper  than  stocking  all  ATE 
arts),  but  it  fails  in  wartime,  particularly  for  the  Air  Force.  In  war,  the 
dr  Force  plans  to  deploy  a  three-squadron  tactical  air  wing  with  three  sets 
if  ATE  to  two  locations,  one  of  the  locations  would  be  assigned  two  squadrons 
'and  the  associated  ATE).  The  squadron  assigned  to  the  other  location  would 
:hen  be  supported  by  just  the  one  set  of  ATE. 

Compatibility.  To  be  effective,  both  the  ATE  and  the  items  to  be 
nested  must  be  compatible.  If  prime  equipment  design  proceeds  without 
letailed  specification  of  the  ATE,  the  logical  consequence  is  incompatibility. 
?his  incompatibility  creates  eitaer  a  need  for  special-purpose  ATE  (contribut- 
.ng  to  proliferation)  or  complex  interface  devices  to  use  the  standard  ATE 
'contributing  to  excessive  TPS  run  times).  The  Navy's  S-3A  aircraft  provides 
in  example  of  the  latter  situation.  To  overcome  the  incompatibility  between 
S-3A  assemblies  and  VAST,  the  Navy  had  to  develop  a  complex  interface  device 
:o  slow  down  the  electronic  clock  speed  of  the  assemblies  to  match  that  of 
/AST.  The  clock  speeds  were  matched,  but  TPS  run  time,  in  turn,  increased. 
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ime  losses,  with  most  line-replaceable  units  requiring  a  unique  interface 
evice;  low  TPS  quality  as  indicated  by  excessive  run  times  and  ambiguity  of 
ault-isolation  resolution;  and  a  much  higher  workload  (numbers  of  UUTs  to  be 
irocessed)  than  estimated  on  the  basis  of  reliability  parameters  of  the  sup- 
lorted  weapon  systems.  To  compensate,  the  Military  Departments  either  field 
idditional  ATE  or  off-load  work  to  other  ATE. 


Durability.  ATE  is  extremely  fragile  equipment,  sensitive  to  tem¬ 
perature,  humidity,  dust,  shocks,  and  quality  of  electric  power  supply.  Yet, 
inderstandably ,  the  Military  Departments  continue  to  stress  the  use  of  that 
equipment  in  environments  in  which  those  factors  are  extremely  difficult  to 
control.  To  illustrate,  field-level  ATE  in  the  Army  may  be  deployed  in  truck- 
mounted  vans  or  in  trailer-mounted  shelters.  The  operational  environment  for 
those  vans  and  shelters  is  severe,  characterized  by  movement  over  rough 
terrain,  frequent  relocation  (at  least  in  wartime),  and  inconsistent  power 
supply  and  air  conditioning.  It  is  not  surprising  that  the  Army  has  had 
considerable  difficulty  developing  ATE  to  be  effective  in  the  field. 

Reliability,  Maintainability,  and  Availability.  The  reliability  of 
currently  fielded  ATE,  as  measured  by  mean  time  between  failures  (MTBF),  is 
relatively  low  compared  to  their  workloads  and  to  the  typical  run  times  for 
TPSs.  For  example,  the  ATE  stations  of  the  F-15  AIS  each  have  an  MTBF  of 
35  to  40  hours;  the  reliability  of  the  MSM-105s  is  in  the  same  range;  and  the 
VAST,  at  least  for  aircraft  carrier  installations,  has  an  MTBF  of  40  to 
50  hours.  Low  reliability,  however,  does  not  necessarily  translate  directly 
into  low  operational  availability  since  no  single  TPS  requires  all  of  the 
stimulus  and  measurement  capabilities  of  the  ATE.  Thus,  even  with  one  or  more 
failures  in  test  instruments,  the  ATE  is  still  capable  of  executing  many  TPSs 
as  long  as  the  computer  and  core  modules  are  operational.  The  maintenance 


(the  length  of  time  varies  by  manufacturer) .  Even  though  work-around  proce¬ 
dures  are  developed,  the  ATE  itself  then  becomes  very  difficult  to  support. 

The  combined  effect  of  these  three  obsolescence  factors  —  engineer¬ 
ing  changes,  equipment  wearout,  and  discontinued  commercial  equipment  —  is  a 
relatively  short  economic  life  for  ATE.  The  Air  Force's  on-going  replacement 
of  the  AISs  for  its  F-15  and  F-16  aircraft  illustrates  this  situation  clearly. 
Those  AISs  were  originally  fielded  in  the  mid-  to  late-1970's. 

Operational  Suitability 

The  requirements  or  specifications  for  ATE  normally  are  stated  in 
terms  of  performance  or  capability  characteristics,  such  as  number  of  input/ 
output  pins,  vector  pattern  depth,  clock  rate,  source/sink  current,  and 
program  logic  level  resolution.  Occasionally,  the  Military  Departments 
acquire  ATE  that  satisfies  those  technical  characteristics,  yet  fails  to 
provide  the  required  support.  Some  of  the  contributing  factors  are  discussed 
below. 

Complexity.  Much  of  the  DoD's  inventory  of  ATE  consists  of  second- 
generation  ATE,  with  a  "rack  and  stack"  architecture  comprising  many  bays  of 
test  instruments  and  designed  to  be  capable  of  testing  a  wide  variety  of 
equipment.  The  result  is  extremely  complex  ATE.  Typical  examples  include  the 
Air  Force's  F-15  AIS,  which  is  more  complex  than  the  weapon  system  itself,  and 
the  Navy's  VAST.  The  implications  of  the  high,  inherent  complexity  of  ATE  are 
low  reliability  and  poor  maintainability. 

Throughput  Capacity.  The  productive  throughput  of  ATE  stations  at 
the  intermediate  level  is  often  less  than  originally  anticipated  or  planned. 
Factors  contributing  to  this  situation  include:  low  reliability  and  inade¬ 
quate  maintainability  of  the  ATE;  queuing  problems  (e.g.,  both  VAST  and  EQUATE 
are  single-port  systems,  capable  of  serving  only  one  UUT  at  a  time);  hook-up 


but  a  number  of  other  factors  have  contributed.  Three  are  briefly  discussed 
below. 


Prime  Equipment  Engineering  Changes .  Engineering  changes  or  equip¬ 
ment  upgrades  are  frequent  in  the  electronics  subsystems  of  prime  weapon 
systems.  Even  though  the  weapon  system  itself  may  have  a  lifetime  of 
30  years,  its  electronics  are  probably  updated  every  4  to  7  years.  Whenever 
that  updating  occurs,  the  original  ATE  invariably  needs  to  be  redesigned  or 
replaced.  In  the  past,  redesign  of  ATE  was  complicated  by  a  lack  of  standard¬ 
ization  in  test  software  languages,  ATE  operating  software,  and  ATE  design 
architecture.  With  the  introduction  of  fourth-generation  ATE,  ATE  redesign 
will  become  an  easier  task  because  prime  equipment  upgrades  will  be  supported 
without  replacing  the  entire  ATE. 

Hardware  Wearout.  Even  if  prime  equipment  engineering  changes  can 
be  accommodated  without  major  ATE  design  changes,  that  option  is  often  imprac¬ 
tical  because  the  ATE  already  has  exceeded  its  design  life.  It  is  common  for 
the  Military  Departments  to  use  ATE  to  such  an  extent  that  its  design  life 
(typically,  20,000  operating  hours)  is  reached  in  5,  rather  than  20,  years. 
This  need  to  operate  available  ATE  so  intensely  occurs  because  (1)  the  ATE 
cannot  provide  the  required  throughput  capacity  or  (2)  the  required  number  of 
ATE  units  to  support  peacetime  operations  are  never  procured. 

Discontinued  Commercial  Equipment.  ATE  normally  is  composed  of  a 
mixture  of  commercial  test  instruments  and  instruments  designed  to  military 
specifications,  with  the  former  being  increasingly  used  because  of  their  cost 
and  schedule  advantages  over  the  latter.  Commercial  test  instruments, 
however,  are  typically  in  production  only  a  short  time  before  being  dropped 
from  the  manufacturer's  catalog.  Within  a  few  years,  the  spares  and  repair 
parts  for  those  out-of-production  test  instruments  are  frequently  unavailable 


have  tended  to  select  specially  designed  test  equipment.  The  result  has  been 
a  proliferation  of  ATE,  with  little  compatibility. 

Attempts  to  control  the  proliferation  of  ATE  often  have  created  more 
problems  than  they  have  cured.  In  the  early  1970's,  the  Air  Force  developed 
the  General  Purpose  Automatic  Test  System  as  its  standard  depot-level  ATE  for 
testing  shop-replaceable  units.  For  a  variety  of  reasons  (technical  as  well 
as  resistance  by  program  managers),  this  program  did  not  succeed.  In  1972, 
the  Naval  Air  Systems  Command  (NAVAIR)  adopted  the  AN/USM-247  Versatile 
Avionics  Shop  Tester  (VAST)  as  its  standard  for  both  intermediate  and  depot 
level.  That  decision,  however,  soon  resulted  in  many  unforeseen  problems, 
including  the  limited  productive  throughput  of  VAST,  the  incompatibility  of 
VAST  with  many  of  the  units  that  it'  was  to  support,  and  the  fact  that  the  size 
and  complexity  of  VAST  made  it  too  difficult  to  support.  As  a  result  of  those 
problems,  the  Navy  was  forced  to  develop  additional  ATE  to  augment  VAST. 

The  Army's  first  major  attempt  at  standardizing  its  ATE  occurred 
several  years  after  those  of  the  Air  Force  and  Navy,  but  with  similar  results. 
In  1979,  the  Army  adopted  the  AN/USM-410  EQUATE  as  the  standard  ATE  for  both 
general  support  and  depot  maintenance.  At  the  general -support  level,  the  Army 
plans  called  for  development  of  two  mobile,  van-mounted  versions  of  EQUATE: 
the  MSM-105(V)1  for  communications-electronics  equipment  and  the  MSM-105(V)2 
for  avionics  (such  as  that  aboard  the  AH-64  helicopter).  When  the  first 
MSM-105(V)1  vans  were  fielded  in  the  fall  of  1983,  representing  early  1970’s 
technology,  the  Army  began  to  experience  many  of  the  same  problems  that  the 
Air  Force  and  Navy  faced  several  years  earlier. 

Obsolescence 


Rapid  obsolescence  is  a  serious  problem  with  ATE,  even  more  so  than 
with  ETE.  One  factor,  the  rapid  evolution  in  ATE  technology,  has  dominated, 


consists  of  portable  testers  that  differ  from  each  other  primarily  in  the 
degree  of  automation  and  the  extent  of  manual  intervention  required.  Many  of 
the  testers  currently  in  use  require  extensive  operator  interaction  and, 
therefore,  probably  should  be  considered  special-purpose  ETE,  not  ATE. 
However,  when  the  checkout  functions  are  fully  automated,  such  as  in  the 
Army's  family  of  Simplified  Test  Equipment  for  internal  combustion  engines  and 
combat  vehicles,  then  the  equipment  clearly  qualifies  as  ATE. 

Off-line  ATE  is  used  principally  at  intermediate-level  activities  and 
depots.  Typically,  it  is  used  to  test  and  fault-isolate  modules,  assemblies, 
and  components  that  have  been  removed  from  the  prime  equipment. 

A  third  way  of  categorizing  ATE,  particularly  off-line,  is  by  interface 
technology.  Currently,  the  UUT  is  stimulated  and  the  response  to  that  stimu¬ 
lation  is  measured  through  edge  connectors,  or  input/output  pins.  With 
earlier  ATE,  however,  a  different  technique,  known  as  "bed  of  nails,"  was 
popular.  (It  is  still  used  extensively  in  plant  checkout  equipment.)  This 
technique  probes  the  circuitry  within  the  DWT  directly,  rather  than  indirectly 
via  the  input/output  pins. 

Most  of  the  problems  associated  with  ATE  are  tied  directly  to  the  rapid 
advancement  in  technology  that  has  taken  place  in  the  past  25  years.  This 
advancement  has  resulted  in  many  different  types  of  ATE  in  Military  Department 
inventories  (although  all  three  have  initiated  major  standardization 
programs);  much  of  the  ATE  is  obsolete;  and  much  of  it  is  not  suitable  for  its 
planned  operational  environment. 

Proliferation  of  ATE 

Most  of  the  ATE  in  the  DoD's  inventory  has  been  acquired  as  support 
equipment  for  major  weapon  systems,  with  the  program  manager  of  the  prime 
equipment  responsible  for  developing  ATE  requirements.  The  program  managers 


TABLE  2-1.  CHARACTERISTICS  OF  SUCCESSIVE  GENERATIONS  OF  ATE 
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The  main  reason,  today,  why  the  Military  Departments  continue  using 
military  specifications  for  general-purpose  ETE  is  their  desire  to  exercise 
complete  configuration  control.  This  makes  the  job  of  managing  the  test 
equipment  and  its  life-cycle  support  easier  but  invariably  results  in  higher 
costs  in  the  short-run  and  greater  technical  obsolescence  in  the  long-run. 
AUTOMATIC  TEST  EQUIPMENT 

The  use  of  ATE  by  the  Military  Departments  has  been  the  focus  of  numerous 
studies,  dating  from  the  early  1970's.  Many  of  those  studies  were  initiated 
because  of  the  effect  that  ATE  has  on  weapon  system  readiness  and  because  of 
its  high  cost. 

Within  the  past  25  years,  four  different  generations  of  ATE  have  been 
developed,  with  each  succeeding  generation  being  more  capable,  more  effective, 
more  efficient,  and  more  supportable  than  its  predecessor.  (Table  2-1  des¬ 
cribes  the  characteristics  of  each  of  the  four  generations.)  Most  of  the  ATE 
in  the  DoD's  inventory  is  primarily  second  generation,  with  some  third  genera¬ 
tion  equipment  such  as  the  Air  Force's  avionics  intermediate  shop  (AIS)  for 
the  F-16,  some  members  of  the  Navy's  ''standard  ATE  family,"  and  the  Army's 
AN/USM-410  EQUATE,  which  is  early  third  generation.  Fourth-generation  ATE 
will  be  introduced  shortly  by  the  Air  Force,  under  its  Modular  Automatic  Test 
Equipment  (MATE)  Program,  but  it  will  be  several  years  before  the  Army  and 
Navy  are  in  a  similar  position. 

Another  way  of  categorizing  ATE  is  whether  it  is  used  on-line  or  off¬ 
line.  On-line  ATE  is  primarily  used  by  organizational-level  mechanics.  It  is 
connected  directly  to  the  prime  equipment,  essentially  to  supplement  the 
built-in-test  (BIT)  and/or  in  lieu  of  manual  ETE.  On-line  ATE  usually 


This  terminology  differs  from  that  used  by  many  people  where  on-line  ATE 
is  considered  to  be  BIT  hardware/software  embedded  in  the  prime  equipment. 


commercial  ETE  may  become  obsolete  from  a  support  perspective  soon  after  the 
manufacturer  stops  producing  it,  primarily  because  spare  parts  are  no  longer 
readily  available.  For  out-of-production  ETE  built  to  military  specifica¬ 
tions,  it  may  be  relatively  easy  for  a  Military  Department  to  find  a  source 
for  spare  parts  and  repair  because  it  has  all  the  needed  specifications. 
Eventually,  however,  even  that  equipment  cannot  be  supported  economically. 
Both  types  of  support  obsolescence  are  common  within  the  DoD. 

Unnecessary  Requirements 

The  Military  Departments  routinely  specify  ETE  performance  and 
support  requirements  that  are  not  necessary  and  inflate  the  cost  of  acquiring 
and  supporting  that  equipment. 

Much  of  DoD's  ETE  continues  to  be  built  to  military  specifications, 
which  stress  high  reliability  and  performance  in  a  hostile  environment.  This 
practice,  however,  assumes  that  military-designed  equipment  is  inherently  more 
reliable  than  commercial  ETE.  This  assumption  may  have  been  well  founded 
several  years  ago,  but  it  is  no  longer  valid.  Furthermore,  the  requirement 
for  operating  the  ETE  in  a  hostile  environment  appears  sometimes  to  be  more  of 
desire  than  of  need. 

In  addition  to  building  much  of  its  ETE  to  military  specifications, 
the  DoD  also  stresses  that  it  be  maintainable  by  military  personnel.  This 
requirement  is  in  marked  contrast  to  that  of  the  commercial  sector,  for  which 
many  ETE  manufacturers  design  their  equipment  to  be  repaired  in  factories  or 
specially  equipped  repair  centers.  As  a  result,  coranercial  ETE  is  much  less 
likely  than  military  ETE  to  embody  functional  partitioning  of  components, 
accessibility,  and  other  design-for-maintenance  features.  Commercial  ETE  is, 
however,  less  costly  because  those  features  add  substantially  to  both  ETE 


design  and  production  costs. 


calibration  requirements  have  not  changed.  The  Air  Force  believes  that  this 
new  equipment  will  perform  the  calibration  faster  and,  over  the  long-term, 
will  be  more  economical  than  existing  equipment.  The  manual  calibration 
equipment  becomes  technically  obsolete  with  the  first  delivery  of  the  newer 
equipment . 

Special-purpose  ETE  is  frequently  rendered  technically  obsolete  when 
weapon  systems  are  modified  to  increase  performance  or  to  expand  capabilities. 
If  those  modifications  introduce  new  requirements  for  special-purpose  ETE,  the 
old  test  equipment  no  longer  can  be  used. 

One  of  the  primary  reasons  that  general-purpose  ETE  becomes  tech¬ 
nically  obsolete  is  that  budget  requests  for  replacement  equipment  are  not 
adequately  supported.  Since  this  equipment  does  not  have  a  strong  consti- 
tutency  with  the  DoD,  it  is  not  uncommon  for  those  requests  to  be  routinely 
reduced  or  even  eliminated.  Eventually,  however,  replacement  equipment  must 
be  procured.  The  Air  Force  recently  faced  such  a  situation  with  its  precision 
measurement  equipment  laboratories.  The  Air  Force  had  not  updated  the  equip¬ 
ment  in  those  laboratories  for  many  years,  so  much  of  it  was  obsolete.  As  a 
consequence,  the  Air  Force  was  forced  to  quickly  establish  a  substantial 
equipment  upgrade  program  for  its  laboratories. 

General-purpose  ETE  actually  becomes  obsolete  very  quickly.  The 
average  production  life  of  this  type  of  equipment  in  commercial  markets  is 
5  to  7  years,  and  when  a  new  model  is  developed,  the  production  of  the  old 
model  is  often  terminated  immediately.  If  that  equipment  is  in  the  DoD's 
inventory,  then  problems  in  supporting  it  surface  quickly. 

Support  Obsolescence.  An  item  of  ETE  is  obsolete  from  a  support 
point  of  view  when  spare  parts,  technical  manuals,  training,  depot  repair,  or 
other  support  no  longer  can  be  economically  obtained.  As  noted  above, 
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overall  effect  of  these  failures  is  a  steady  stream  of  new  and  unique  ETE 
being  procured  by  the  Military  Departments. 

Acquisition  Practices.  The  second  major  contributor  to  the  pro¬ 
liferation  of  ETE  within  the  DoD  is  the  frequency  with  which  the  Military 
Departments  give  a  prime  contractor  extensive  latitude  in  selecting  and  devel¬ 
oping  the  ETE  to  support  a  major  weapon  system.  It  is  common  for  a  Military 
Department  to  establish  a  line  for  ETE  in  the  basic  contract  without  specify¬ 
ing  the  types  of  ETE  that  will  be  provided,  the  quantity  of  each  type 
required,  and  when  the  ETE  will  be  delivered.  Those  specifics  are  then  nego¬ 
tiated  after  development  of  the  weapon  system  has  begun,  at  great  disadvantage 
to  the  Government  both  in  terms  of  managing  costs  and  controlling  the  prolif¬ 
eration  of  ETE.  Similarly,  reprocurement  of  a  specific  type  of  test  equipment 
is  often  opened  to  competition,  resulting  in  a  growing  mix  of  different  makes 
and  models  of  the  same  type  of  test  equipment. 

Obsolescence 

Within  the  DoD,  ETE  becomes  obsolete  when  it  is  technically  outdated 
or  when  it  no  longer  can  be  supported.  Both  forms  of  obsolescence  are  more 
common  with  general-purpose  ETE  than  with  special-purpose  ETE. 

Technical  Obsolescence.  Technical  obsolescence  occurs  when  new 
requirements  for  test  equipment  cannot  be  met  by  ETE  already  in  the  inventory. 
An  example  would  be  a  military  mechanic  attempting  to  troubleshoot  a  digital 
computer  system  with  test  equipment  incapable  of  measuring  the  time  intervals 
needed.  Although  this  example  of  technology  transcending  available  ETE  is 
common  (as  well  as  expected)  within  the  DoD,  technical  obsolescence  of  ETE  is 
also  driven  by  changes  in  requirements  «nd  operating  environments.  As  an 
example,  the  Air  Force  recently  initiated  a  program  to  replace  much  of 
its  manual  calibration  equipment  with  automated  equipment,  even  though  the 


To  place  the  proliferation  in  even  more  concrete  terms,  the  Army,  as 
part  of  its  TMDE  Modernization  Program,  bought  10  hew  types  of  general-purpose 
ETE  in  FY82.  Those  lu  replaced  650  different  makes  and  models  in  the  Army's 
inventory.  Additional  purchases  of  other  types  of  general-purpose  ETE  are 
planned  over  a  10-year  time  frame. 

The  causes  of  ETE  proliferation  are  numerous,  but  many  can  be  traced 
to  shortcomings  in  the  validation  of  technical  requirements  and  in  the 
practices  followed  in  acquiring  new  ETE. 

Technical  Requirements.  A  common  form  for  specifying  ETE  require¬ 
ments,  particularly  for  aviation  weapon  systems,  is  the  Support  Equipment 
Recommendation  Data  (SERD) .  The  prime  contractor  for  the  weapon  system 
usually  prepares  the  SERD  and  then  forwards  it  to  a  military  activity  that 
specializes  in  reviewing  requirements  for  support  equipment.  That  activity 
ascertains  that  the  requirements  for  new  general-purpose  ETE  cannot  be  satis¬ 
fied  by  items  already  in  the  inventory  and  that  the  requirements  for  special- 
purpose  ETE  are  valid. 

In  practice,  this  process  fails  to  control  the  proliferation  of  ETE 
because  (1)  the  support  equipment  activities  are  given  only  60  days  to  review 
each  SERD,  a  period  that  in  many  cases  is  inadequate  (the  B-l  aircraft  alone 
generated  9,000  SERDs;  annually,  the  Air  Force  processes  more  than 
30,000  SERDs);  (2)  many  SERDs  are  incomplete  and  thus  require  additional 
attention  from  activity  personnel  (the  Air  Force  estimates  that  35  percent  of 
its  SERDs  are  not  properly  completed) ;  (3)  the  reviewing  activities  have 
neither  an  up-to-date  listing  of  preferred  ETE  nor  the  authority  to  require 
that  already  available  ETE  be  used;  and  (4)  prime  contractors  (and  the  program 
managers)  routinely  submit  SERDs  for  special-purpose  ETE  or  new  general- 
purpose  ETE  irrespective  of  the  ETE  already  in  the  DoD's  inventory.  The 
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2.  OVERVIEW  OF  TEST  EQUIPMENT  PROBLEMS 


This  chapter  summarizes  DoO's  problems  with  test  equipment,  with  separate 
treatment  for  ETE,  ATE,  and  TPSs. 

ELECTRONIC  TEST  EQUIPMENT 

Many  of  the  DoD's  problems  with  ETE  were  first  given  extensive  visibility 
by  a  Defense  Science  Board  Task  Force  in  1976.  During  a  2-year  period,  the 
task  force  reviewed  the  DoD's  acquisition  and  management  of  ETE  and,  based  on 
the  results  of  that  review,  developed  28  recommendations  to  correct  observed 
problems.  Even  though  several  of  the  recommendations  were  fully  implemented, 
many  of  the  important  problems  identified  by  the  task  force  still  remain:  the 
DoD's  inventory  of  ETE  consists  of  too  many  different  line  items  (i.e.,  makes 
and  models  of  ETE),  with  much  of  it  obsolete  and/or  designed  to  satisfy 
unnecessary  requirements. 

Proliferation 

The  proliferation  of  ETE  within  the  DoD  can  be  best  illustrated  by 

comparing  the  number  of  different  makes  and  models  of  ETE  with  the  minimum 

number  of  different  types  required.  Table  1-1  shows  that  the  Army  has  more 

than  18,000  different  makes  and  models  of  general-purpose  ETE.  Yet,  one 

source  concluded  that  the  Army  has  a  requirement  for  only  600  types  of 

general-purpose  ETE.1  Similarly,  in  1982  the  Navy  concluded  that  its  total 

2 

requirement  for  general-purpose  ETE  was  fewer  than  200  types,  not  the 
29,000  different  makes  and  models  shown  in  Table  1-1. 


XLTG  J.  M.  Heiser,  Jr.,  USA  (Ret.),  ''Assessment  of  DA/DARCOM  TMDE 
Program,"  September  1979. 

^Department  of  the  Navy,  "Standard  General  Purpose  Electronic  Test 
Equipment,"  MIL-STD-1364F,  1  March  1982. 
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by  contractors  are  typical,  especially  for  TPSs  used  at  the  intermediate- 
maintenance  level.  For  some  systems,  the  costs  associated  with  updating  TPSs 
are  deemed  excessive  and  the  updates  are  not  procured.  As  an  example,  the 
Air  Force  is  not  updating  the  TPSs  for  F-15  line-replaceable  units,  developed 
at  a  cost  in  excess  of  $500  million,  because  of  the  cost.  (To  reduce  program 
acquisition  costs,  the  Air  Force  did  not  procure  the  technical  data  for  F-15 
line-replaceable  units.  Without  that  data,  it  is  impossible  to  identify  why 
TPS  diagnostic  performance  is  not  satisfactory  and  to  make  necessary  improve¬ 
ments.)  Another  factor  contributing  to  this  decision  may  be  that  those  units 
are  inadequately  designed  for  testability.^ 

The  problems  of  late  development  and  inadequate  performance  of  TPSs  are 
caused  by  many  factors,  but  three  appear  to  dominate:  the  complexity  of 
developing  TPSs,  the  inherent  limitations  of  acceptance  tests,  and  the  absence 
of  management  tools  to  monitor  TPS  performance  and  cost. 

Complexity  of  TPS  Development 

The  development  of  a  TPS  is  a  complex  process.  To  compensate  for 
that  complexity,  the  Military  Departments  frequently  do  not  adhere  to  estab¬ 
lished  TPS  development  practices  and  standards  and  permit  simplifying  assump¬ 
tions  that,  on  the  surface,  save  both  time  and  money.  Over  the  long  run, 
however,  those  short-term  savings  result  in  the  fielding  of  TPSs  that  are  not 
available  in  a  timely  manner,  are  not  effective,  and  are  excessively  costly. 

Development  of  a  TPS  begins  with  the  UUT  test  specifications. 
Establishing  those  specifications  is  a  laborious,  long,  costly,  and  labor- 
intensive  task.  It  is  not  unusual  for  weapon  system  contracts  to  be  modified 

**Poor  design  for  testability  of  the  prime  equipment  is  one  of  the  root 
causes  of  high  cost  and  poor  performance  of  currently  fielded  TPSs.  The 
F-15’s  problems  with  testability  are  documented  in  "Minutes  of  the  F-15  RTOK 
Management  Group  Meeting,"  Warner  Robins  Air  Logistics  Center/  Support  Systems 
Associates,  Inc.,  November  1982. 
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to  waive  many  of  the  provisions  for  UUT  test  specifications.  While  this 
practice  reduces  the  cost  of  the  basic  contract,  it  increases  the  difficulty 
of  using  organic  resources  to  maintain  and/or  improve  the  TPS  once  it  is 
developed,  because  the  required  technical  data  are  not  available. 

Not  only  are  the  UUT  test  specifications  costly  to  develop,  the  DoD 
does  not  yet  have  the  tools  and  methods  to  verify  and  validate  them.  As  a 
result,  the  DoD  must  rely  on  the  contractor's  assertion  that  the  specifica¬ 
tions  are  accurate  and  complete. 

Following  development  of  UUT  test  specifications,  the  contractor 
then  designs  the  test  program,  test  program  instructions,  and  interface 
devices  necessary  to  satisfy  the  specifications.  To  condensate  for  some  of 
the  complexity  of  that  design,  the  Military  Departments  frequently  permit 
contractors  to  use  several  simplifying  assumptions.  Those  assumptions  often 
limit  the  fault  detection  capability  of  the  TPS  to  single,  "hard”  failures 
that  occur  within  electronic  components.  Furthermore,  failures  caused  by  poor 
workmanship,  design  deficiencies,  incorrect  assembly  in  the  factory,  acci¬ 
dental  damage,  and  UUT  cannibalization  are  normally  excepted  from  the  TPS 
design  logic.  Thus,  many  of  the  failures  that  would  be  encountered  in  an 
operational  environment,  such  as  wiring  failures,  multiple  failures,  inter¬ 
mittent  failures  in  digital  modules,  or  cooq>onents  damaged  from  improper 
handling,  often  cannot  be  fault- isolated  by  the  TPS. 
limitations  of  TPS  Acceptance  Tests 

Acceptance  testing  of  TPSs,  whether  developed  in-house  or  under 
contract,  is  considered,  by  the  Military  Departments,  to  be  just  another 
standard  quality  assurance  responsibility.  The  test  essentially  consists  of 
selecting  a  small  sample  of  possible  UUT  failures,  inserting  them  one-by-one 


into  a  UUT  test  article,  executing  the  TPS  on  the  ATE  in  accordance  with  test 


program  instructions,  and  observing  the  TPS's  performance  in  detecting  and 
isolating  each  failure  accurately  within  specified  levels  of  ambiguity.  If 
the  TPS  passes  the  test,  it  is  formally  accepted.  Copies  are  then  made  for 
distribution  to  maintenance  activities  and  the  TPS/UUT  data  are  entered  into  a 
data  base  for  configuration  management. 

This  type  of  TPS  acceptance  test  is  inadequate  however.  A  UUT  may 
possess  tens  of  thousands  of  different  failure  modes,  many  of  which  are  not 
associated  with  the  10  or  20  failures  that  are  inserted  for  the  purposes  of 
the  test.  Furthermore ,  the  sample  failures  are  se  acted  from  a  contractor- 
prepared  fault  list. 

It  is,  of  course,  impractical  to  conduct  a  complete,  physical  test 
of  a  TPS  for  a  complex  UUT.  The  DoD  does  not  yet  have  the  tools  or  the 
methods  to  verify  TPS  performance  through  other  means  (simulation) .  Until 
such  time,  the  Military  Departments  must  recognize  that  TPS  acceptance  testing 
represents,  at  best,  a  necessary  but  not  sufficient  tool  for  verifying  TPS 
performance;  rather,  monitoring  the  performance  of  fielded  TPSs  appears  to  be 
an  essential  requirement. 

Absence  of  Management  Tools 

The  Military  Departments  have  not  developed  the  management  tools 
necessary  to  ensure  that  their  large  investments  in  TPS  development  and  sup¬ 
port  result  in  the  fielding  of  high-quality  TPSs.  The  costs  of  developing  and 
supporting  TPSs  typically  are  not  available,  while  the  management  systems  for 
monitoring  TPS  performance  are  inadequate.  The  common  explanation  for  the 
former  is  that  the  cost  data  are  not  required,  while  for  the  latter,,  the 
current  procedures  for  collecting  and  reporting  TPS  deficiencies  do  not  work 


for  test  program  software. 


One  of  the  shortcomings  frequently  attributed  tc  the  current  proce¬ 
dures  for  reporting  TPS  deficiencies  is  that  they  cannot  adequately  isolate 
deficiencies  attributable  to  operator  inexperience,  ATE,  ATE  control  software, 
UUT,  interface  devices,  and  TPS.  The  consequence  is  that  TPS  problems  are 
routinely  submerged,  denying  TPS  developers  and  supporters  the  information 
needed  to  identify  deficiencies  and  correct  them. 

Other  important  indications  of  TPS  performance  that  need  to  be  made 
available  routinely  are  run  time  and  diagnostic  performance  (i.e.,  fault- 
isolation  resolution) .  The  goal  for  a  TPS  developer  is  to  design  a  TPS  that 
always  fault-isolates  the  UUT's  problems  in  the  shortest  possible  time.  Yet, 
unless  the  diagnostic  performances  of  TPSs  and  their  run  times  are  routinely 
recorded,  improvement  in  TPS  quality  will  be  slow. 

In  the  following  chapter,  we  describe  the  numerous  initiatives  that  OSD 
and  the  Military  Departments  have  taken,  or  are  taking,  to  correct  the  prob¬ 
lems  with  test  equipment. 


3.  SYNOPSIS  OF  DoD  INITIATIVES 


This  chapter  summarizes  many  of  the  key  initiatives  taken  by  the  DoD  in 
recent  years  to  improve  test  equipment  management  and  support. 

JOINT  OR  COORDINATED  INITIATIVES 

In  the  late  1970's,  the  Joint  Logistics  Commanders  (JLC)  established  a 
Panel  on  Automatic  Testing  to  develop  and  implement  a  long-range,  definitive 
program  on  automatic  testing.  The  program,  which  was  approved  in  1979, 
comprises  nearly  100  tasks  divided  into  3  categories:  (1)  management, 

(2)  acquisition  support,  and  (3)  testing  technology.^  The  efforts  of  that 
panel  have  resulted  in  the  publication  of  several  Joint-Service  Guides  (e.g., 
"Automatic  Testing  Acquisition  Planning  Guide,"  a  "Built-in-Test  Design 
Guide,"  and  a  "Selection  Guide  for  Digital  Test  Program  Generation  System") 
and  the  development  of  improved  automatic  testing  policies  and  procedures. 
The  panel's  efforts  have  also  resulted  in  the  initiation  of  several  tasks, 
most  of  which  are  still  ongoing,  in  the  management  and  acquisition  support 
categories . 

In  response  to  the  findings  of  the  1976  Defense  Science  Board  Study  of 

ETE,  the  Deputy  Under  Secretary  of  Defense  (Acquisition  Policy)  tasked  the 

Defense  Materiel  Specifications  and  Standards  Office  (DMSSO)  in  1980  to 

develop  plans  and  procedures  for  reducing  the  proliferation  of  general-purpose 
2 

ETE.  To  carry  out  that  tasking,  DMSSO  established  the  Ad  Hoc  DoD  TMDE 
Standardization  Group,  comprised  of  representatives  of  the  Under  Secretary  of 


"Joint  Agreement  on  Support  of  the  Automatic  Testing  Program,”  signed  by 
the  DARCOM/NMC/AFLC/AFSC  Commanders,  16  July  1979. 

^Deputy  Under  Secretary  of  Defense  (Acquisition  Policy) ,  Memorandum  for 
the  Director,  DMSSO,  Subject:  "Test,  Measurement,  and  Diagnostic  Equipment 
(TMDE),  FSC  6625,"  13  May  1980. 
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Defense  for  Research  and  Engineering  (USDR&E)  and  the  Military  Departments. 
The  primary  emphasis  of  that  group  has  been  on  eliminating  military  specifica¬ 
tions  for  obsolete  test  equipment.  It  recently  issued  a  status  report  on  its 

3 

standardization  efforts  and  is  scheduled  to  publish  a  standardization  plan  in 
1985. 

Also  in  1980,  the  JLC  established  a  panel  to  examine  the  impediments  to 
increased  standardization  of  aviation  ground  support  equipment  (AGSE) .  The 
final  report  of  the  panel,  which  was  issued  in  1983,  called  for  specific 
improvements  in  the  areas  of  policies  and  procedures,  data  storage  and 
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retrieval,  and  contract  methods  and  specifications.  The  panel's  recommenda¬ 
tions  included: 

-  Establish  AGSE  interservice  coordination  positions  in  each  Military 
Service; 

-  Revise  procedures  for  contractor  review  of  AGSE  data  bases; 

-  Encourage  AGSE  acquisition  managers  to  make  standardization  a  major 
technical  factor  in  source  selection; 

-  Use  Military-Handbook-300  as  the  primary  data  storage  and  retrieval 
system  for  ground  support  equipment  identification  and  selection. 

Although  specifically  focused  on  AGSE,  many  of  the  panel’s  recommendations  are 

also  applicable  to  other  areas  of  test  equipment.  In  approving  the  panel's 

report,  the  JLC  agreed  to  implement  most  of  the  recommendations  "within  each 

of  our  Commands  to  the  maximum  reasonable  extent  as  resources  permit." 

In  1982,  the  National  Security  Industrial  Association  (NSIA)  established 

an  Automatic  Testing  Committee  to  support  and  supplement  the  JLC  Panel  on 

Automatic  Testing.  Many  of  the  tasks  undertaken  by  the  committee  were  in 
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Department  of  the  Army,  United  States  Army  Communications 
nd,  Standardization  Program  Analysis,  29  June  1984. 
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Joint  DARCOM/NMC/AFLC/AFSC  Commanders  Panel  on  Standardization  of  Avia¬ 
tion  Ground  Support  Equipment,  Final  Report.  28  June  1983. 


response  to  the  recommendations  of  an  earlier  Industry/DoD  study  of  automatic 
testing  issues,  completed  in  1980. 

Each  of  the  above  initiatives  addressed  acquisition  or  inventory  manage¬ 
ment  of  test  equipment.  Coordinated  action  with  respect  to  test  equipment 
support  dates  back  to  1967,  when  the  JLC  established  the  Joint  Technical 
Coordinating  Group  for  Metrology  and  Calibration.  That  group  was  tasked  to 
promote  increased  standardization  in  the  calibration  practices  of  the  Military 
Departments.  It  achieved  some  successes  in  the  consolidation  of  duplicative 
calibration  facilities  —  an  effort  that  began  in  1975  with  the  establishment 
of  a  separate  subgroup  for  consolidation  of  calibration  services.  This  sub¬ 
group  conducted  studies  of  the  feasibility  and  cost-effectiveness  of  consoli¬ 
dation  on  a  regional  basis.  Some  consolidation  has  occurred  as  a  result,  but 
the  subgroup's  recommendations  for  restructuring  the  calibration  services  in 
Europe  were  never  approved.  Similarly,  recommendations,  emanating  from  out¬ 
side  the  calibration  community,  to  centralize  management  and  institute  a 
Joint-Service  Metrology  and  Calibration  Center  have  not  been  acted  upon. 
MILITARY  SERVICE  INITIATIVES 

Army 

For  the  past  5  years,  the  Army  has  been  in  the  midst  of  an  extensive 
program  to  improve  its  management  and  support  of  TMDE.  In  1979,  it  introduced 
a  new  TMDE  calibration  and  repair  concept.  The  old  concept  was  characterized 
by  (1)  split  responsibility  for  calibration  and  repair  and  (2)  two  levels  of 
field  calibration.  The  new  concept,  selected  as  a  result  of  a  comprehensive 
study, ^  provides  total  TMDE  calibration  and  repair  from  a  single  source  on  an 
area-basis.  It  also  assigns  to  the  Army  Materiel  Command  (AMC)  (previously 

^Department  of  the  Army,  "Concept  Study  for  Improved  Army-Wide  TMDE  Cali¬ 
bration  and  Repair  Operations,"  March  1977. 


DARCOM)  responsibility  for  the  management  and  control  of  all  calibration  and 
repair  of  common  TMDE,  as  well  as  for  selected  special-purpose  TMDE. 

Also  in  1979,  AMC  announced  that  it  would  adopt  the  AN/USM-410 

EQUATE  as  the  Army's  standard  general-purpose  ATE  for  use  at  general  support 

and  depot  levels.  The  policy  for  determining  whether  system-peculiar  ATE 

planned  for,  or  already  in  development  for,  new  weapon  systems  would  be 
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acceptable  in  deviation  from  the  AN/USM-410  EQUATE  was  promulgated  in  1980. 
That  policy,  in  effect,  emphasizes  the  benefits  of  standardization  and  permits 
a  one-time  cost  increase  to  convert  planned  ATE  to  the  AN/USM-410  EQUATE  when 
long-term  cost  or  readiness  benefits  warrant. 

The  decision  to  use  the  AN/USM-410  EQUATE  as  the  standard  ATE  in  the 
Army  was  followed  closely  by  the  establishment  of  the  TMDE  Modernization  Pro¬ 
gram.  That  program,  which  was  developed  jointly  by  the  U.S.  Army 
Communications-Electronics  Command  (CECOM)  and  the  U.S.  Army  Signal  Center, 
was  designed  to  correct  many  of  the  Army's  longstanding  problems  with  TMDE. 
Under  the  program,  the  Army  would  replace  obsolete  manual  ETE  with  commercial 
off-the-shelf  equipment.  In  the  program's  first  year  (FY81),  5  items  (types 
of  test  equipment)  were  procured  (total  count  of  3,600),  replacing  21  dif¬ 
ferent  makes  and  models  in  the  inventory.  In  FY82,  10  items  were  procured, 
replacing  650  makes  and  models.  With  the  subsequent  TMDE  management 
reorganization  in  1982  (see  below) ,  responsibility  for  the  program  was  trans¬ 
ferred  to  the  Product  Manager,  TMDE  Modernization,  and  the  program  expanded  to 
non-CECOM  items. 

In  April  1982,  the  Secretary  of  the  Army  designated  the  Commanding 
General,  AMC,  as  the  Army's  TMDE  Executive  Agent  responsible  for  TMDE  acqui¬ 
sition,  logistics,  and  financial  management,  and  AMC's  Deputy  Commanding 

^Department  of  the  Army,  "Implementation  Plan  for  Single  ATE  (General 
Support  and  Depot)  Policy,"  MIC,  June  1980. 
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General  for  Materiel  Readiness  as  the  Executive  Director  for  TMDE.  These  and 
other  actions  resulted  in  the  Army  establishing  a  centralized  TMDE  management 
structure  consisting  of  three  key  activities: 

-  Centralized  TMDE  Activity,  Lexington,  Kentucky;  responsible  for 
equipment  acquisition  approval  and  TMDE  evaluation,  among  others; 

-  U.S.  Army  TMDE  Support  Group,  Huntsville,  Alabama;  responsible 
for  TMDE  calibration  and  repair,  operation  of  primary  standards 
laboratories,  and  assessment  of  TMDE  supportability; 

-  Program  Manager  for  TMDE,  Fort  Monmouth,  New  Jersey;  responsible 
for  the  TMDE  modernization  program,  development  of  ATE  languages, 
and  TMDE  funding. 

This  centralization  of  the  Army's  traditionally  dispersed  TMDE 
management  structure,  which  had  been  recommended  by  an  earlier  study, ^  was 
recommended  by  a  recent  study  conducted  by  the  Department  of  the  Army  TMDE 
Action  Team  (DATAT),  which  was  undertaken  at  the  request  of  the  Assistant 
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Secretary  of  the  Army  (Installations,  Logistics,  and  Financial  Management). 
The  DATAT  report,  besides  recommending  a  centralized  management  structure, 
identified  numerous  other  actions  for  improving  the  Army's  management  and 
support  of  test  equipment.  Those  recommendations  were  approved  and  a  plan  to 
monitor  Army-wide  implementation  was  established.  Several  of  the  specific 
initiatives  undertaken  as  a  result  of  the  DATAT  report  are  identified  below: 

-  Policy.  In  April  1984,  the  Army  updated  its  TMDE  policies  by 
issuing  Army  Regulation  750-43,  "Test,  Measurement  and  Diagnostic 
Equipment."  This  regulation  provides  improved  procedures  for 
requirements  identification,  selection,  acquisition,  and  life- 
cycle  support  for  all  types  of  TMDE. 

-  Systems  Approach.  Traditionally,  the  Army  has  acquired  and 
fielded  TMDE  on  an  incremental  basis.  Now,  the  Army  is  deter¬ 
mining  the  mission  requirements  of  its  maintenance  organizations 
and  the  best  mix  of  TMDE  to  meet  those  requirements.  The  Program 


^LTG  J.  M.  Heiser,  Jr.,  USA  (Ret.),  "Assessments  of  DA/DARCOM  Test, 
Measurement,  and  Diagnostic  Equipment  Program,"  September  1979. 

g 

Department  of  the  Army  Test,  Measurement  and  Diagnostic  Equipment  Action 
Team  (DATAT)  Final  Report,  March  1982. 
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Manager,  TMDE,  is  pursuing  this  approach,  starting  in  FY85  with 
the  High  Technology  Motorized  Division  stationed  at  Fort  Lewis, 
Washington.  A  reduction  in  overall  numbers  of  TMDE  is  antici¬ 
pated  from  this  approach. 

-  Test  Equipment  Modernization  Program.  This  program  was  ori¬ 
ginally  focused  on  FSC  6625  test  equipment  for  which  CECOM  is  the 
inventory  control  point.  In  September  1983,  it  was  extended  to 
other  FSCs  but  the  funding  was  not  increased. 

*  TMDE  Support.  The  Army  recently  completed  a  comprehensive  study 
of  TMDE  support,  including  automatic  calibration,  which  was  man¬ 
dated  by  Congress.9 

Another  Army  initiative,  with  a  much  broader  impact  than  just  TMDE, 
was  to  change  its  maintenance  doctrine.  In  1982,  the  Army  eliminated  general 
support  maintenance  units  from  the  corps  area  and  replaced  them  with  inter¬ 
mediate  (rear)  units  located  behind  the  corps  rear  boundary.  This  change  has 
had  a  far-reaching  impact  on  the  Army's  ATE  requirements.  With  the  inter¬ 
mediate  (rear)  units  requiring  little  mobility,  the  deployment,  operational 
use,  and  support  of  the  field-level  version  of  the  AN/USM-410  EQUATE  has 
become  more  feasible.  At  the  same  time,  the  doctrine  mandates  highly  mobile 
ATE  for  assembly  testing  and  repair  by  intermediate  (forward)  maintenance 
units  (formerly  direct  support)  in  corps,  division,  and  forward  areas. 

Marine  Corps 

The  primary  test  equipment  initiative  of  the  Marine  Corps  is  to 
develop  its  own  ATE  standardization  program.  That  program,  which  is  scheduled 
to  enter  full-scale  development  in  1985,  emphasizes  modular  architecture  and 
commercial  off-the-shelf  test  instruments,  using  a  standard,  commercial  data 
bus.  The  Marine  Corps  is  planning  to  field  the  new  ATE  at  all  maintenance 
echelons  requiring  ATE  for  fault  isolation  of  assemblies  and  modules  after 
they  are  removed  from  the  end  item.  It  will  be  configured  as  bench-top  or 
rack-mounted  equipment  and  installed  in  maintenance  vans. 
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Deputy  Executive  Director  for  TMDE,  "Final  Comprehensive  Report  on  the 
U.S.  Any  Calibration  Program,"  Department  of  the  Army,  July  1984. 


Navy  initiatives  to  control  or  reduce  the  proliferation  of  general- 
purpose  ETE  date  to  1969,  when  the  Naval  Material  Command  (NAVMAT)  delegated 
centralized  management  responsibility  for  general-purpose  ETE  to  the  Naval 
Electronic  Systems  Command  (NAVELEX) .  In  exercising  its  authority,  however, 
NAVELEX  was  hampered  by  the  lack  of  agreement  on  what  constitutes  general- 
purpose  ETE.  As  a  result,  much  of  the  Navy  continued  to  procure  ETE  under  the 
guise  of  special-purpose  ETE  beyond  NAVELEX  control.  NAVMAT  attempted  to 
solve  this  problem  in  1973  by  establishing  an  ETE  Board,  responsible  for 
determining  whether  ETE  is  either  general  or  special  purpose. The  ETE  Board 
eventually  developed  a  standard  item  list  (MIL-STD-1364  (Navy),  "Standard 
General  Purpose  Electronic  Test  Equipment"),  with  NAVELEX  responsible  for 
keeping  the  list  up  to  date  and  restricting,  whenever  possible,  new  general- 
purpose  ETE  procurements  to  items  on  that  list.  It  also  developed  a  special 
approval  process  for  use  when  requirements  dictate  deviating  from  the  standard 
item  list  —  MIL-STD-1387  (Navy),  "Procedures  for  the  Acquisition  of 
Non-Standard  General  Purpose  Electronic  Test  Equipment." 

A  recent  initiative  by  NAVELEX  was  the  establishment  of  a  "General 
Purpose  ETE  Assets  Screening  Pool"  program,  which  consists  of  a  central 
registry  of  Fleet  assets  in  excess  of  allowance  for  redistribution.  This 
program  was  established  in  response  to  a  General  Accounting  Office  report 
which  noted  that,  with  the  exception  of  a  limited  Naval  Sea  Systems  Command 
(NAVSEA)  program,  the  Navy  had  no  such  capability. **  More  recently,  NAVELEX 

*®NAVMAT  Instruction  5430.52,  "Electronic  Test  Equipment;  classification 
and  assignment,"  10  May  1973,  implemented  by  NAVELEX  Instruction  5420.12, 
"Naval  Material  Command  Electronic  Test  Equipment  Classification  Board; 
policies  and  procedures,"  26  October  1973  (Revision  A,  dated  21  April  1976). 

^General  Accounting  Office,  "Survey  of  DoD's  Management  of  Automatic  and 
General-Purpose  Electronic  Test  Equipment  (LCD-80-106),"  Letter  Report  to  the 
Secretary  of  Defense,  B- 199353,  September  4,  1980. 


ias  begun  installing  an  automated  general-purpose  ETE  data  base.  The  first 
rersion  of  that  data  base,  excerpted  from  the  more  comprehensive  Metrology 
Automated  System  for  Uniform  Recall  and  Reporting  (MEASURE)  data  base,  was 
Installed  in  mid-1984,  but  it  is  incomplete  (NAVELEX  estimates  that  30  percent 
>f  the  Navy's  inventory,  based  on  cost,  is  not  included  in  the  data  base). 
NAVELEX  is  now  enhancing  this  data  base. 

Navy  initiatives  with  regard  to  automatic  testing  date  to  the  mid- 
1970' s  when  the  Assistant  Secretary  of  the  Navy  for  Research  and  Development 
requested  that  a  study  be  conducted  to  define  the  problems  and  develop  solu¬ 
tions  of  the  Navy's  use  of  ATE.  This  study  was  conducted  by  representatives 

of  NAVMAT,  Systems  Commands,  field  activities,  Fleet,  and  industry.  The 
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resulting  report,  known  as  the  "Marcy  Report,"  identified  numerous  problems 
and  developed  a  variety  of  recommendations  to  solve  them,  including: 

-  Establishment  of  a  central  ATE  management  group  within  NAVMAT, 
reporting  directly  to  the  Chief  of  Naval  Material. 

-  Education  of  management  personnel  in  the  technical  and  management 
issues  involved  in  weapon  system  acquisition,  including  the 
practical  problems  of  BIT  and  off-line  ATE  hardware/software. 

-  Provision  of  quick  relief  to  the  Fleet  by:  (1)  initiating  engi¬ 
neering  changes  (reliability  improvements)  for  high-failure  items 
of  prime  equipment  as  well  as  ATE,  (2)  establishing  "tiger-teams" 
to  respond  to  Fleet  ATE  problems,  (3)  developing  organic  test 
programming  capabilities,  and  (4)  prohibiting  deployment  of 
off-line  ATE  without  prior  approval  for  Service  use. 

-  Development  of  a  new  family  of  general-purpose  ATE  and  institute 
policy  requiring  the  Chief  of  Naval  Material  approval  of  any 
off-line  ATE  acquisition. 

-  Initiation  of  research  and  development  programs  in  automatic 
testing  technology. 

In  addition  to  this  in-house  effort,  the  Navy  also  requested  an  independent 
assessment  of  its  ATE  efforts  from  industry.  The  Industry  Ad  Hoc  ATE  Project 
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Assistant  Secretary  of  the  Navy  (Research  and  Development),  Report  on 
Navy  Issues  Concerning  Automatic  Test,  Monitoring  and  Diagnostic  Systems  and 
Eauioment.  February  1976. 


is  chartered  in  November  1975,  and  its  final  report  was  published  in 
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?ril  1977.  This  effort,  in  turn,  was  extended  into  the  Industry/ Joint 
ervices  Automatic  Test  Project  when  the  Navy  recognized  that  its  problems  in 
xploiting  automatic  testing  existed  DoD-wide.  As  noted  earlier  in  this 
hapter,  the  latter  study  was  completed  in  June  1980,  with  many  of  the 
ecommendations  pursued  by  joint  working  groups  under  auspices  of  the  JLC 
utomatic  Testing  Panel  and  the  NSIA  Automatic  Testing  Committee. 

The  Navy  began  implementing  most  of  the  actions  recommended  in  the 
Marcy  Report"  in  1976.  A  Test  and  Monitoring  Systems  Project  Office  was 
stablished,  with  responsibility  for  reviewing  acquisition  projects;  conduct- 
ng  coordinated  research  and  development  efforts  in  testing  technology; 
leveloping  ATE  policies,  procedures,  and  management  tools;  and  establishing  a 
.ist  of  "approved  ATE."  In  a  1980  NAVMAT  reorganization,  that  project  office 
/as  transferred  to  NAVE1EX. 

NAVAIR,  in  assessing  the  lessons  learned  from  VAST,  formulated  its 
ITE  plans  for  the  1980's  and  beyond  and  documented  them  in  the  NAVAIR  ATE  Pro¬ 
gram  Plan  (dated  January  1978,  approved  and  promulgated  in  1979).  The  goals 
>f  that  plan  included: 

-  Integrate  ATE  program  management; 

-  Improve  ATE  acquisition; 

-  Design  avionics  for  testability  and  maintainability; 

-  Minimize  the  variety  of  ATE; 

-  Consolidate  and  improve  ATE  software; 

-  Improve  the  quality  of  TPSs; 

-  Attain  full  and  timely  organic  support  capability. 
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key  goal  was  to  minimize  the  variety  of  ATE,  with  the  ultimate  objective 
ng  an  ATE  inventory  with  standardized,  modular  hardware  and  software 
ments  using  a  single,  standard  test  language.  To  achieve  that  goal,  the 
n  defined  a  functional  family  of  common  ATE  to  be  used  in  the  1980's,  and 
cribed  a  new  ATE  project,  the  Consolidated  Support  System  (CSS),  for  use  in 
:  1990's  and  beyond. 

The  system  definition  phase  of  the  CSS  program  started  with  contract 
irds  to  five  competing  contractor-teams  in  January  1982  and  was  completed  in 
fust  1983.  The  Navy  has  completed  its  evaluation  of  the  contractor's 
forts  and  anticipates  releasing  a  request  for  proposal  in  January  1985,  with 
itract  award  for  full-scale  development  expected  in  mid-1985.  NAVAIR’s 
rategy  emphasizes  supportability ,  modularity,  technology  transparency  (i.e., 
signed  for  technology  updating),  rapid  reconfigurability,  operational  evalu- 
ion  of  prototype,  and  ability  to  compete  the  production  phase,  with 
liveries  starting  in  1992.  The  total  CSS  Program  is  estimated  to  cost 
.7  billion. 

NAVSEA's  initiatives  in  automatic  testing  are  centered  in  the 
pport  and  Test  Equipment  Engineering  Program  (STEEP) .  Initiated  in 
tober  1978,  STEEP's  goal  is  to  improve  Fleet  support  of  electronic  modules 
d  printed  circuit  boards  (PCBs)  by  using  digital  card  testers  at 
ganization-  and/or  intermediate-maintenance  activities  to  identify  PCBs  that 
n  be  repaired  in  the  field.  A  subsequent  field  test  of  this  concept,  con- 
cted  from  1979  to  1981,  resulted  in  (1)  the  AN/USM-465  being  selected  as  the 
andard  intermediate- level  ATE  for  testing  digital  PCBs  and  (2)  repairs  of 
lected  PCBs  being  authorized  at  the  intermediate  level.  Additional  field 
sts,  completed  in  1983,  resulted  in  NAVSEA  extending  authorization  of 
gital  card  testers  and  some  PCB  repairs  to  the  organizational  level. 


Between  July  1981  and  July  1984,  the  Navy  established  75  STEEP  sites 
(51  shipboard,  24  ashore),  procured  100  AN/USM-465s  digital  card  testers,  and 
developed  and  installed  about  430  unique  TPSs  (total  density  10,500).  In 
FY85,  the  number  of  STEEP  sites  is  scheduled  to  increase  to  110,  the 
AN/USM-465  inventory  to  several  hundred,  and  the  number  of  unique  TPSs  to 
1,200  (total  density  of  20,000);  the  annual  growth  of  unique  TPSs  is  projected 
to  be  300  to  5 00  over  the  next  5  to  7  years.  To  assist  in  the  management  and 
support  of  these  TPSs,  the  Navy  established  an  ATE/TPS  Coordination  Center  to: 
(1)  provide  a  point  of  contact  for  all  TPS  users,  (2)  maintain  ATE/TPS 
configuration  and  deployment  status  accounting,  (3)  process  TPS  trouble 
reports,  and  (4)  provide  management  information.  The  center  is  located  at  the 
Fleet  Analysis  Center,  Corona,  California,  with  computer  facilities  for 
deployment  and  configuration  information  and  for  TPS  maintenance/development 
tools . 


Among  the  many  activities  of  the  ATE/TPS  Coordination  Center,  it 
distributes  quarterly  to  each  ship  the  Catalog  of  Automatic  Testing  Capabilit 


for  Electronic  Modules/Printed  Circuit  Boards,  identifying  the  prime  equipment 


and  the  specific  modules  for  which  TPSs  are  currently  available  and  the 
specific  test  sites  involved.  It  also  publishes  the  Master  Test  Program  Set 
Index,  listing  the  available  TPSs  by  ATE-type  and  weapon  system.  Furthermore, 
it  supports  use  of  the  Hierarchical  Integrated  Test  Simulator,  a  new  digital 
autooiatic  test  program  generator  that  was  developed  by  the  Naval  Air 
Engineering  Center  with  contractor  support.  That  simulator  is  planned  as  the 
standard  Navy  digital  automatic  test  program  generator  for  the  future;  it 
provides  more  capability  than  previous  generators  used  by  the  Navy,  especially 
with  regard  to  large-scale  and  very-large-scale  integrated  circuits. 


Air  Force 


In  1976,  the  Air  Force,  in  response  to  its  longstanding  problems 
with  ATE  (proliferation  of  test  systems,  inadequate  fault  detection  and  isola¬ 
tion,  escalating  acquisition  costs,  short  life  cycles,  low  system  avail¬ 
ability,  and  a  lack  of  corporate  memory  of  lessons  learned),  established  the 
MATE  Program  as  a  systematic  approach  to  the  acquisition  of  ATE  to  support 
future  weapon  systems.  The  concept  of  the  MATE  Program  was  based  upon  using 
standard  ATE  architecture,  interfaces,  and  software  such  that  different  ATE 
stations  could  be  configured  from  a  common  set  of  ATE  modules,  and  recon¬ 
figured  to  accommodate  weapon  system  modifications.  The  concept/validation 
phase  for  the  MATE  Program  was  conducted  from  June  1978  to  June  1981,  with 
two  competing  contractors  developing  a  uniform  ATE  architecture  with  standard 
interfaces  and  "intelligent'*  stimulus/measurement  instruments  and  the 
management/technical  tools.  This  phase  resulted  in  a  set  of  MATE  Guides  that 
provide  the  standards,  specifications,  and  procedures  for  implementing  each 
contractor's  concept: 

-  Electronic  Test  Equipment  Acquisition  Guide; 

-  MATE  Development  Guide; 

-  Avionics  Testability  Design  Guide; 

-  Production  and  Operational  Support  Guide; 

-  Test  Program  Set  Acquisition  Guide. 

In  July  1981,  the  full-scale  engineering  development  contract 
($55  million)  was  awarded  to  refine  the  MATE  Guides,  develop  the  MATE  data 
base  system,  and  demonstrate  MATE  in  two  ways:  (1)  by  applying  MATE  concepts 
to  the  development  of  the  intermediate  automatic  test  system  for  the  A- 10 
inertial  navigation  system,  with  the  contractor  responsible  for  ATE 
integration,  and  (2)  by  providing  technical  support  to  the  Air  Force  for  the 


development  of  the  Depot  Automatic  Test  System  for  Avionics,  with  the 
Air  Force  responsible  for  ATE  integration. 

The  MATE  Program,  apart  from  its  importance  to  the  Air  Force,  has 
had  a  significant  influence  on  the  other  Military  Departments  as  well.  It  has 
encouraged  a  systematic  management  approach  to  the  acquisition  and  support  of 
ATE  as  a  support  system,  not  a  stand-alone  item.  Furthermore,  major  portions 
of  the  MATE  Guides  are  under  review  for  adoption  as  Joint-Service  Guides  under 
the  JLC  Automatic  Testing  Panel.  Over  SO  percent  of  the  JLC's  automatic 
testing  program  budget  for  the  years  1978  through  1983  (the  budget  totaled 
$250  million  for  those  years)  was  MATE-related. 

The  Air  Force's  commitment  to  MATE  was  demonstrated  by  its  publica- 
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tion  of  the  implementing  regulation  in  early  1984.  The  regulation  applies 
to  all  Air  Force  Systems  Command  (AFSC)  and  Air  Force  Logistics  Command  (AFLC) 
organizations  that  acquire,  modify,  replace,  and  support  weapon  systems.  The 
regulation  includes  the  policy,  organizational  structure,  waiver  process,  and 
authority  required  to  make  MATE  the  standard  Air  Force  ATE.  It  implements  the 
use  of  the  detailed  guides,  control  and  support  software,  and  automated  tools 
produced  by  the  MATE  Program.  It  also  establishes  a  MATE  Operations  Center  to 
perform  MATE  qualification  testing  for  hardware  and  software  modules  and  to 
provide  access  to  the  MATE  automated  management  tools  and  training  system. 
The  MATE  Operations  Center,  established  at  the  San  Antonio  Air  Logistics 
Center,  Texas,  is  also  responsible  for  configuration  control,  maintenance,  and 
distribution  of  MATE  guides,  specifications,  standards,  and  other 
documentation. 

— 

Air  Force  Systems  Command  (AFSC)  and  Air  Force  Logistics  Command 
(AFLC),  "Policy  for  Modular  Automatic  Test  Equipment  (MATE),"  AFSC/AFLC  Regu¬ 
lation  800-23,  25  January  1984. 


In  1979,  AFLC  identified  a  need  for  information  on  what  ATE  was 
deployed  and  where  and  how  it  was  being  utilized.  The  system  capability  and 
data  automation  requirements  for  such  a  system  were  submitted  and  approved  in 
1980.  Development  of  that  system,  Test  Equipment  Reporting  and  Management 
System  (TERMS),  was  completed  in  1981,  and  pilot-implementation  began  in 
mid-1982  at  five  air  bases.  The  system  consists  of  a  centralized  data  base 
that  is  updated  on-line  by  the  air  bases,  which  report  all  transactions 
involving  installed  ATE.  The  formatted  input  data  include  ATE  utilization 
records,  ATE  status  records,  and  station  inventory  records.  Worldwide  imple¬ 
mentation  of  TERMS  is  awaiting  Air  Staff  authorization. 

The  San  Antonio  Air  Logistics  Center  is  in  the  process  of  modifying 
TERMS  to  interface  with  other  systems  (such  as  the  C104  ATE  data  bank  that 
contains  information  on  ATE  inventory  and  operating  parameters,  the  D039  Item 
Manager’s  inventory  asset  balance  for  equipment  items,  and  the  MATE  data 
bank).  When  completed,  the  modified  TERMS  will  provide  Major  Air  Force 
Commands  access  to  the  system  (currently,  only  the  individual  bases  have 
access  to  it)  and  use  of  several  software  models. 

In  1980,  the  Aerospace  Guidance  and  Metrology  Center  (AGMC)  found 
that  the  Air  Force's  Precision  Measurement  Equipment  Laboratories  (PMELs)  had 
problems  calibrating  and  repairing  test  equipment  and  that  their  backlogs  and 
reliance  on  contractor  support  were  increasing.  In  1981,  AGMC  convened  a 
worldwide  PMEL  conference  to  identify  the  causes  of  those  problems  and  the 
actions  required  to  correct  them.  Some  of  the  corrective  actions  that 
occurred  as  a  result  of  that  conference  included: 

-  Policies/Plans.  Additions  were  made  to  the  War  and  Mobilization 
Plan  which  required  planning  for  intermediate-maintenance  support 
of  test  equipment  (previously,  the  assumption  had  been  that  the 
nearest  PMEL  could  provide  support,  not  recognizing  that  all 
PMELs  are  different). 
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-  Equipment .  To  reduce  the  obsolescence  of  PMEL  equipment,  a  sig¬ 
nificant  increase  in  funding  for  the  replacement  of  that  equip¬ 
ment  was  programmed  in  the  POM. 

-  Procedures .  Procedures  were  developed  to  improve  item  management 
of  PMEL  standards  and  to  reduce  the  amount  of  test  equipment 
fielded  without  calibration  requirements  documentation;  AGMC- 
prepared  calibration  procedures  were  reorganized  and  the  proce¬ 
dure  verification  process  was  tightened. 

-  Personnel/Training .  Proposals  (still  under  consideration)  were 
developed  to  improve  retention  and  increase  the  experience  level 
of  PMEL  personnel. 

Another  AGMC  initiative  was  to  develop  a  better  approach  for  cali¬ 
brating  ATE.  Traditionally,  the  Air  Force  calibrated  ATE  by  removing  the 
tester-replaceable  units  from  the  ATE  at  specified  intervals  and  then  sending 
those  units  to  a  PMEL  for  manual  calibration.  This  approach  had  many  disad¬ 
vantages,  including: 

-  ATE  downtime  was  significant;  the  ATE  was  down  any  time  a  tester- 
replaceable  unit  was  removed,  thus,  the  ATE  was  seldom  completely 
assembled. 

-  Programmable  features  could  not  be  exercised  in  calibration; 
timing  and  time-related  problems  with  the  ATE  were  either  masked 
or  ignored,  so  that  those  units  that  function  on  the  test  bench 
might  not  work  in  the  ATE. 

-  System  performance  was  not  checked;  no  compensation  or  allowance 
for  cabling,  loads,  switching,  or  other  sources  of  signal 
degradation  between  the  tester-replaceable  units  and  the  UUT  was 
possible. 

-  The  tester-replaceable  units  were  subject  to  damage  in  transit 
between  the  ATE  site  and  the  PMEL. 

-  High  levels  of  integration  in  the  ATE  made  off-site  calibration 
difficult  or  impossible;  many  measurement/stimulus  functions 
previously  performed  by  individual  tester-replaceable  units  were 
often  implemented  at  the  card  level.  Removal  and  calibration  of 
those  functions  in  the  PMEL  was  difficult. 

-  The  tester-replaceable  units  were  not  calibrated  in  their  opera¬ 
tional,  physical,  and  electrical  environment. 


To  overcome  the  shortcomings  of  off-site  calibration,  AGMC  developed 
an  on-site  calibration  concept,  the  Portable  Automatic  Test  Equipment 
Calibrator,  with  the  following  features: 

-  Portable  calibration  standards  are  brought  in  protective  cases  to 
the  ATE  site. 

-  The  ATE  is  calibrated  as  a  total  system,  addressing  only  the 
required  functions  to  required  accuracies. 

-  Calibration  is  performed  as  close  to  the  UUT  interface  as 
possible. 

-  PMEL  personnel  perform  the  ATE  calibration  with  user  (ATE 
operators)  assistance  and  participation. 

-  Host  standards  are  programmable,  providing  as  totally  automated  a 
calibration  procedure  as  possible. 

Between  1976  and  1984,  the  Air  Force  procured  210  portable  calibrators,  with 
most  assigned  to  PMELs. 

In  response  to  problems  associated  with  transferring  program  manage¬ 
ment  responsibility  from  AFSC  to  AFLC  for  major  weapon  systems  during  the 
1970's,  the  Air  Force  established  in  1982  a  Deputy  Chief  of  Staff  for  Acquisi¬ 
tion  Logistics  at  AFSC  Headquarters.  The  responsibilities  of  that  office 
include  management  of  acquisition  logistics,  product  assurance,  standardiza¬ 
tion,  computer  resources,  and  support  equipment  and  automatic  testing  policy. 
AFLC  then  established  a  single  system  manager  for  automatic  test  systems  at 
San  Antonio  Air  Logistics  Center  and  designated  it  the  AFLC  focal  point  for 
readiness  and  logistics  support  of  ATE.  Responsibilities  of  the  automatic 
test  system  manager  include  planning  for  logistics  support  (in  coordination 
with  the  weapon  system  program  office  and  its  deputy  program  manager  for 
logistics),  screening  of  the  inventory  for  equipment  capable  of  satisfying  new 
testing  requirements,  evaluating  SERDs,  and  managing  all  fielded  ATE. 

Deputy  program  managers  for  logistics  have  been  assigned  to  AFSC 
system  program  offices  since  the  early  1970's.  To  enhance  the  role  of 


acquisition  logistics  and  its  potential  for  influencing  weapon  system  design, 
the  Air  Force  recently  redesignated  the  Air  Force  Acquisition  Logistics 
Division  (an  AFLC  field  organization  within  AFSC,  including  the  deputy  program 
managers  for  logistics)  as  a  joint  AFSC/AFLC  organization,  known  as  the 
Air  Force  Acquisition  Logistics  Center.  Further  changes  in  management 
structure,  policies,  and  procedures  can  be  anticipated  as  a  result  of 
continued  dissatisfaction  with  the  way  support  equipment  is  managed,  evidenced 
by  a  recently  completed  Air  Force  study. 

In  March  1984,  the  Support  Equipment  Acquisition  Review  Group  was 
established  at  the  request  of  the  Assistant  Secretary  of  the  Air  Force 
(Research,  Development,  and  Logistics)  and  formally  chartered  by  the  AFSC  and 
AFLC  Commanders  to  study  the  entire  support  equipment  acquisition  process.  In 
its  final  report,  which  was  issued  in  July  1984,  the  Review  Group  identified 
19  major  problems  in  support  equipment  acquisition  and  made  107  recommenda¬ 
tions  to  solve  them.  In  assessing  the  operational  impacts  of  the  support 
equipment  problems,  it  concluded  that  shortages  in  support  equipment  (esti¬ 
mated  at  a  cumulative  value  of  $1.5  billion)  do  not  impact  peacetime  mission 
accomplishment  because 

.  .  .  extensive  workarounds  and  personal  ingenuity  are 
being  used  to  accomplish  the  mission,  but  that  these 
shortages  could  potentially  have  mission  impacts  if  we  are 
stressed,  and  particularly  if  we  are  stressed  in  more  than 
one  direction  at  the  same  time.15 

The  recommendations  of  the  Review  Group  have  been  approved  by  the  Air  Force 
Secretariat,  and  the  Air  Staff  has  instituted  a  tracking  system  to  monitor 
their  implementation. 


3 Support  Equipment  Acquisition  Review  Group  Final  Report,  July  1984. 
The  report  defines  support  equipment  as  comprising  two  categories:  TMDE 
(comprising  precision  measurement  equipment,  ATE  hardware,  ATE  software,  TPSs, 
and  special  test  equipment)  and  ground  support  equipment. 


SUMMARY 


The  preceding  synopsis  of  DoD  initiatives  to  alleviate  the  shortcoming  in 
test  equipment  management  and  support  is  neither  complete  nor  exhaustive. 
Nevertheless,  it  demonstrates  that  the  Military  Departments  are  well  aware  of 
the  shortcomings  and  have  already  launched  programs  to  correct  many  of  them. 
The  synopsis  also  documents  the  near  absence  of  OSD  participation  in  the 
initiatives,  which  we  believe  has  resulted  in  some  of  the  initiatives  being 
less  successful  than  anticipated.  In  the  following  chapter,  we  set  forth 
several  specific  actions  OSD  should  take  to  solve  some  of  DoD's  fundamental 
management  problems  with  test  equipment. 


4.  RECOMMENDED  ACTIONS 


SYNOPSIS 

The  DoD  has  a  loag  history  of  problems  in  acquiring  and  supporting  test 
equipment.  The  OSD,  the  Military  Departments,  and  industry  associations  have 
sponsored  numerous  studies,  task  forces,  inspections,  and  panels  to  document 
the  problems,  identify  causes,  and  recommend  solutions.  Some  of  those  efforts 
focused  on  particular  technical  issues,  others  addressed  broader,  DoD-wide 
issues.  Regardless  of  their  focus  or  sponsorship,  however,  they  have 
routinely  documented  the  same  problems. 

The  DoD  has  too  many  different  types  of  ETE,  and  the  procedures  for 
controlling  further  inventory  growth  are  not  effective.  Much  of  that  equip¬ 
ment  is  obsolete,  and  so  it  is  difficult  to  support.  It  is  also  excessively 
costly  because  of  requirements  that  much  of  it  be  built  to  military  specifica¬ 
tions,  rather  than  to  use  commercial  design  standards. 

The  DoD  has  not  effectively  controlled  the  growth  of  ATE.  Furthermore, 
much  of  that  equipment  suffers  from  rapid  obsolescence,  insufficient  through¬ 
put  capacity,  and  inadequate  performance. 

The  DoD  has  difficulty  acquiring  TFSs  in  a  timely  manner.  Many  are  not 
as  effective  as  planned,  and  most  require  excessive  run  times  to  fault-isolate 
problems.  All  need  to  be  updated  within  a  few  years  of  initial  fielding. 

In  response  to  those  problems,  the  OSD  and  the  Military  Departments  have 
undertaken  numerous  corrective  efforts.  Those  that  have  focused  on  the  acqui¬ 
sition  process  show  some  long-term  potential.  Greater  emphasis  is  being 
placed  on  support  during  weapon  system  development,  procurement  of  test  equip¬ 
ment  i'^  being  delayed  until  weapon  system  designs  stablize,  and  standardized 
ATE  is  being  developed. 
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Many  of  the  corrective  efforts,  however,  have  resulted  in  the  establish- 
aent  of  another  Joint  Service  study  group  or  panel,  usually  to  examine  a 
specific  technical  issue.  Those  examinations  have  had  a  tendency  to  be  very 


lengthy  and  seldom  have  resulted  in  decisive  actions. 

The  few  efforts  that  have  addressed  the  management  and  support  of  test 
equipment  after  it  enters  the  inventory  have  been  launched  with  considerable 
enthusiasm.  All  too  often,  however,  they  have  lacked  the  support,  either 
within  a  Military  Department  or  throughout  the  DoD,  that  is  required  to  carry 
out  their  original  objectives. 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  DoD  continues  to  be  plagued  with  test  equipment  problems  because 
(1)  the  associated  technical  issues  have  been  and  continue  to  be  difficult  to 
overcome,  (2)  the  responsibility  for  test  equipment  management  within  the 
Military  Departments  is  fragmented,  and  (3)  the  OSD  has  not  provided  the 
leadership  in  effecting  improvements  in  the  management  and  support  of  fielded 
test  equipment. 

For  the  past  several  years,  USDR&E,  working  both  with  the  Military 
Departments  and  the  JLC,  has  been  spearheading  DoD's  efforts  to  overcome  the 
technical  problems.  Those  efforts,  many  of  which  are  now  on  the  verge  of 
yielding  significant  benefits,  have  focused  primarily  on  changes  to  the  acqui¬ 
sition  process.  They  also  have  included  the  funding  of  research  and  develop¬ 
ment  programs  to  resolve  specific  technical  problems. 

Since  the  early  1980's,  each  of  the  Military  Departments  has  established 
some  type  of  single  manager  for  test  equipment.  For  the  most  part,  however, 
those  single  managers  have  been  assigned  responsibility  for  just  one  small 
segment  of  test  equipment,  usually  general-purpose  ETE.  The  responsibility 
for  special-purpose  ETE,  for  example,  continues  to  be  fragmented  among 
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numerous  program  and  system  managers.  As  a  consequence,  the  Military  Depart¬ 
ments  have  made  some  progress  toward  improved  test  equipment  management,  but 
additional  improvements,  many  of  which  are  substantial,  are  still  required. 

The  primary  effect  of  OSD's  lack  of  involvement  in  the  management  and 
support  of  test  equipment  has  been  to  reinforce  the  Military  Departments' 
perception  that  test  equipment  is  not  a  high  priority  item.  Yet,  the  DoD's 
investment  of  approximately  $30  billion  in  test  equipment  and  the  effect  of 
that  equipment  on  weapon  system  readiness  and  sustainability  makes  that  posi¬ 
tion  no  longer  tenable.  To  the  contrary,  those  factors  suggest  substantial 
and  immediate  involvement. 

We  believe  that  the  efforts  of  the  USDR&E  and  the  Military  Departments  to 
overcome  the  technical  problems  with  test  equipment  need  to  be  augmented  with 
complementary  actions  focusing  on  the  management  and  support  of  fielded  test 
equipment. 

Furthermore,  we  believe  that  the  Assistant  Secretary  of  Defense 
(Manpower,  Installations,  and  Logistics),  ASD(MI&L),  must  be  the  catalyst  to 
effect  those  actions  within  the  DoD.  We  recommend  that  he  charge  the 
Director,  Maintenance  Policy,  to  take  the  following  actions  fundamental  to  an 
effective  test  equipment  management  program  within  the  DoD: 

-  Develop  a  DoD-wide  preferred  items  list  for  ETE,  as  well  as  procedures 
to  assure  that  program  and  item  managers  use  the  items  on  that  list  to 
satisfy  ETE  requirements; 

-  Institute  reporting  of  ATE  performance  and  availability; 

-  Establish  the  collection  of  TPS  run  times  and  performance; 

-  Draft  a  DoD  instruction  that  prescribes  policy  and  procedures  for 
managing  and  supporting  test  equipment. 

These  actions  constitute  a  small,  first  step  toward  improving  management 
and  support  of  test  equipment  within  the  DoD.  The  first  action,  development 
and  use  of  a  preferred  items  list,  is  the  most  effective  approach  to  reducing 


le  proliferation  of  ETE  within  the  DoD.  The  ramifications  of  such  a  list  can 
i  far-reaching,  from  reducing  ETE  procurement  costs  to  improving  ETE  support, 
le  second  and  third  actions  are  designed  to  enable  the  Military  Departments 
o  identify  shortfalls  in  the  performance  of  fielded  ATE  and  TPSs.  Currently, 
be  Military  Departments  do  not  have  formal  procedures  for  collecting  such 
nformation.  The  fourth  action  fills  a  void  in  DoD  policy  that  is  partially 
esponsible  for  many  of  the  current  inadequacies  in  test  equipment  management 
nd  support. 

The  Director,  Maintenance  Policy,  certainly  cannot  take  the  actions  we 
econnnend  in  isolation.  He  will  need  to  work  closely  with  representatives  of 
he  USDR&E  and  the  Military  Departments  to  effect  even  the  most  straightfor¬ 
ward  action.  Nevertheless,  he  will  offer  a  fresh  perspective  on  what  needs  to 
le  accomplished  and  how  to  do  so  most  effectively.  He  will  also  be  in  posi- 
;ion  to  ensure  that  the  Military  Departments  follow  through  with  their  commit- 
lents  and  programs  to  improve  the  management  and  support  of  fielded  test 
equipment . 
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